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U — Abst r ac t — 

Two simulation and analysis environments have been developed to support telerobotics research at 
the Langley Research Center. One is a high-fidelity, nonreal-tlme, interactive model called ROBSIM, 
which confines user-generated models of workspace environment, robots, and loads into a working system 
and simulates the interaction among the system components. Models Include user-specified actuator, 
sensor, and control x *rameters* as well as kinematic and dynamic characteristics. Kinematic* dynamic* 
and response analyses can be selected, with system configuration, task trajectories, and arm states 
displayed using computer graphics. The second environment is a real-time, manned Telerobotic Systems 
Simulation (TRSS1 which uses the facilities of the Intelligent Systems Research Laboratory (ISRL). It 
utilizes a hierarchical structure of functionally distributed computers communicating over both parallel 
and high-speed serial data paths to enable studies of advanced telerobotic systems. Multiple processes 
perform motion planning, operator communications, forward and Inverse kinematics, control /sensor fusion, 
and I/O processing while communicating via common memory. This paper describes both ROBSIM and TRSS, 
including their capability, status, and future plans. Also described is the architecture of ISRL and 
recent telerobotic system studies in ISRL. 

2. Introduction 

Historically, simulation has proven to be a cost-effective method for obtaining estimates of the feasibil- 
ity and/or value of new technologies. This is particularly true where direct experience is difficult or 
impossible to obtain, either because physical systems do not exist and must therefore be nmdelled and 
simulated, or because access to the environment for testing is limited. Both conditions apply for space 

telerobotic systems and tasks such as in space assembly and servicing. 

Two simulation and aculysis environments have been developed to support telerobotics research. One is a 
high-fidelity, nonreal -time, interactive model called ROBSIM, which combines user-generated models of workspace 
environment, robots, and loads into a working system and simulates the interaction among the system 
components. Models include user-specified actuator, sensor, and control parameters, as well as kinematic and 
dynamic characterl sties. Systems consist of up to five (5), ten (10) degree-of-freedom robot arms, on either 
independent or common moving bases, kinematic, (iynamlc, and response analyses can be selected, with system 
configuration, task trajectories, and arm states displayed using computer graphics. 

The second environment is a real-time, manned Telerobotic Systems Simulation (TRSS) which uses tine 
facilities of the Intelligent Systems Research Laboratory (ISRL). It utilizes a hierarchical structure of 
functionally distributed computers communicating over both parallel and high-speed serial data paths to enable 
studies of advanced telerobotic systems. Multiple processes perform motion planning, operator communications, 
forward and inverse kinematics, control /sensor fusion, and I/O processing while communicating via common 
memory. Additional hardware elements of the simulation include symbolic processor, high-speed computer 
graphics system, manned control station, dual PUMA 560 manipulators, and a vision processor. 

This paper describes ROBSIM, TRSS and ISRL, and their relationship to advanced telerobotic system studies 
performed at Langley. 


3. High Fidelity Robotic Simulation 

In 1981 Langley contracted with Martin Marietta Aerospace Corp., for an "Evaluation of Automated Decision- 
making Methodology and Development of an Integrated Robotic System Simulation. The objective was _to bring 
within NASA machine intelligence methodologies in automated decision-making, and to develop a robotic syste* 
simulation as a testbed for applying this and other technologies needed for space robotic systems. The result 
was the identification of artificial intelligence (AI) methods applicable to automated decision making, and a 
framework for an integrated robotic simulation, including manipulator forward and inverse kinematics and 
dynamics [I]. 

Another result of the contract was the Remote C-bltal Servicing System (ROSS) concept which, employing 
state-of-the-art technology, could service the Solar M«r satellite, the Long Duration Exposure Facility, and 
the Space Telescope, to the same extent as man, in EVA or in the Shuttle payload bay [2]. Figure 1 3 * S k° * th * 
ROSS, in a two-arm configuration attached to a conceptual Orbital Maneuvering Vehicle. ROSS later became a 
focus for promoting space robotics within NASA [3], 


293 



Subsequent Improvements to the simulation have Included [4]: 

1. Implementation of a preliminary interface between a computer-aided design (CAD) data base and the 
ROBSIM data base via the Initial Graphics Exchange Standard (IGES) format [5J. 

2. The ability to define the actuator and control system to the detail desired. For example, motor back 
emf and Joint coulomb friction can be modelled, if desired. Either fixed-gain or adaptive control 
systems can be modelled, and response data can be output for analysis. 

3. Extension of the simulation to Incorporate multiple manipulator arms, each having up to 10 degrees of 
freedom, attached to either Independent or common moving bases. 

4. The capability for the user to quickly Interactively define a simple arm geometry for conceptual design 
tasks ,-and then-edit-and reflne-the-arm geometric-and-mass -properties fop-detafled-response 

simulations. 

5. An Interactive "help" feature which allows the user to Interrupt the simulation, ask for and receive 
Information on simulation characteristics and capabilities, and then resume processing. 

6. Rehosting the graphics output to Digital Equipment Corp. (DEC) graphics-compatible terminals, 
enabling a low-cost graphics display capability. 

ROBSIM runs on a DEC VAX 11/750 computer under the VMS operating system. Its speed depends upon the level 
of detail in the system definition and the subsystems simulated. When simulating the dynamics of ' l " u 
manipulator arms, performing path planning or detailed control system modelling, c °"|PUting 
for a kinematically redundant arm (having seven or more degrees of freedom), or when outputting to a low 
performance graphics system, the simulation operates slower than real time, but still at an acceptable Inter- 
active rate. Figure 2 Illustrates steps In using ROBSIM In a telerobotic system development cycle: 

1. Interactive definition of manipulator arms(s) and end effectors, and base to which each arm Is 
attached. 

2. Definition of the static environment. 

3. Definition of dynamic environment features, Including the loads to be carried and manipulated. 

4. Synthesis of the system. 

5. Development of the task description, using a simple task language and elementary task functions. 

6. Simulation of the task. 

7. Graphics display of task plus performance data output to a printer, plotter, or terminal. 

8. Analysis of results by the user and determination of desired changes. 

Despite Its capabl 1 1 tes and ease of use, ROBSIM has three significant limitations. First, Itis essen- 
tially a rigid body simulation. Second, for most applications. It runs slower than real time. Third, It is 
primarily a robotic, rather than telerobotic simulation. 

Several approaches to modelling manipulator structural flexibility have been ^xamlned lncludlng assu^g 
Hold links and modelling bending as a compliant Joint rotation, frequency domain analysis retaining over- 

order modes, and finite element analysis, and symbolic rather than numei r \ c L° f ex^^o" 4 " JltTele^t 

One ootlon being considered Is to provide the user the option of Interfacing to an exls ting finite eleme 

structural analysis program. This would enable the user to control the level of detail desired and provide an 

interface to computer-aided design and analysis programs. 

A promising sppro.ch to spooning up dynamics calculations Is to distribute tj» citation o«r multiple 
Drocessors. The rigid body dynamics equations are recursive and coupled and appear JLo P^ra e 

processing. The application of concurrent processing to manipulator dynamics Is an on-going research act y 

at Langley. 

The distinction between a robotic system and a telerobotic system is important. With a robotic 
huma^can interrupt a task, but usually Interacts .1th the system at the end of a task or 
processes may occur simultaneously, with the human actlrg as a monitor or su P ervi sor, but not as a 
In a telerobotic system, the human can act as a monitor or supervisor of automated tasks, and can also s are 
control with or take control from the robotic controller. Unlike ROBSIM. -here the response of the s Hwlated 
system may be output at rates faster or slower than real time, depending upon the complexity of the system- 

simulated, a telerobotic simulation must reference all timing to real time. 

The ROBSIM program has been provided to over 20 organizations, Including most NASA Ce " ters - ^ 

Force, both aerospace and nonaerospace companies, and several universities. ROBSIM will 
improved and disseminated. 
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4. TeleRobotlc System Simulation (TRSS) 

In order to model and study telerobotlc tasks, a real-time, man-in-the-loop TeleRobotlc System Simulation 
(TRSS) was developed. TRSS enables Investigations of space-related telerobotlc applications through a 

combined hardware and software simulation utilizing the facilities of the Inte ” i 1 9 *", t . S £!! ei Ii Studied* 
Laboratory (ISRL). ISRL provides an environment where teleoperator and robotic technologies may be studied, 
and It enables Integration and evaluation of telerobotlc system and task hardware. 

Development of TRSS began In 1981. In 1982 a computer graphics simulation of a 5 degree-of-freedo« 
manipulator performing a precision alignment task, with time del ay of up to 2 seconds, JJ* TJ? 

simulation ran on a Control Oata Corp. Cyber 175 computer and used displays and 1n t *; 

aircraft cockpit simulator. Results showed that subjects changed from continuous control to a 
strategy for time delays of .25 sec and longer. Time required to perform the simulated task Increased 1 nearly 


Wl VII V » i v/ I V*. » * ■ * 

In 1983. TRSS was rehosted to a VAX 11/750 In the Intelligent Systems Research Laboratory and Interfaced to 
a control station, a general purpose data acquisition system, and a Unimate PUMA 560 manipulator. Subse- 
quently, a vision system and a second PUMA 560 were added. The PUMA typically used In "plck-and-place 
Industrial applications, Is a digitally-controlled six degree-of -freedom (DOF) anthropomorphic "J^P^tor. It 
has been augmented with a parallel Jaw gripper and a six DOF wrist force/torque sensor. ,! JJKITm a 250 
uses an LSI 11-73 computer running the RT-11 operating system. Communication among processors occurs on a 250 
Ubvte/sec ' packet-switching global bus conforming to IEEE Std. 488-1978. The VAX serves as network controller 
and cowron memory. A Hyperchannel Interface provides a direct memory access (DMA) Interface between the Cyber 
"S V Ax" 0 "ompaite r s; a hfgVspeed serial Interface connects the VAX to a Symbolics 3670 computer; and a DMA 
Interface is being Implemented between the VAX and a Rediffusion Computer Graphics Corp. Poly 2000 high-speed 
computer graphics subsystem. 

Figure 3 Illustrates the hierarchical, distributed architecture In the current ISRL c° n ^ 9 “ nation. F ! gur ! 

4 shows a similar architecture for the software modules in TRSS. The mapping of the TRSS modules was developed 
In a manner analogous to manned aircraft or spacecraft systems; the operator will be capable of 
manipulator control strategies and assuming direct manual control at any time. However ^petltlous or tedious 
tasks will be automated so the operator need only monitor or supervise these functions. The modularity of TRSS 
allows the user freedom to utilize as much or as little hardware as desired, and allows modification or 
functional replacement of i.ny software module. 

One of the first telerobotlc studies in ISRL was an active compliance task. Strain gauges mounted in the 
fingers of the end effector sensed constraint forces and torques during close tolerance peg Insertion, and fed 
this data to control and display modules via the data acquisition system. A simple 9 r apb 1 cs dl^lay mg. 5) 
indicated the magnitude and direction of the binding forces and torques. Using the display the curator could 
readily command the arm to move to null any disturbing forces. TRSS was then modified to allow the operator to 
select a mode in which the strain gauge data were fed directly to the control system to automatically null the 
force and torques. The operator then used the display to monitor the automatic Insertion. 

The next step was to Incorporate automated control based on vision sensing. The philosophy In TRSS Is to 
partition vision processing between man and machine, giving each responsibility for that which It does best. 
Man performs image recognition and Interpretation, while the machine vision system performs Image acquisition, 
low-level processing, and determines the location of objects. In ISRL the Image processing system Involves a 
16-level gray-scale imaging system, a 240 X 320 X 4 frame buffer, and processing algorithms on the POP 11-73 
computer. The simulation Is structured such that only those modules which _ actively manipulate the 
system data structures (low-level image processing routines) will require modification If a d1f ^*^ nt 
used. Current research efforts center on determining the three-space location and orientation of labeled 
objects using a single camera, and on a multifunction recognition operator for telerobotlc vision I.7J. 

In 1985. vision-based control and force/torque control were integrated In a simulated satellite s * rv jc1r>9 
task. Figure 6 shows a mockup of a biostack experiment module carried by the Long Duration Exposure Facility 
(LDEF) satellite. Three bolts must be loosened before the simulated module could be removed. The operator s 
task was to position the television camera so as to acquire a labeled object In the f1 f 1d oF (1n . 

four LEDs). After acquisition the operator Invoked the vision system, which determined the location of the 
module with respect to the camera and the end effector, and then commanded the manipulator to move to the 
module. After positioning the end effector above the first bolt, the vision system notified the operator and 
transferred control to the force/torque system, which moved the wrench to the bolt and commanded the end 
effector to torque the bolt for release. 

An advantage of the TRSS control structure, which uses resolved motion commands to the manipulators, is It 
enables the operator to share control with the autonmous system rather than having to switch between manual and 
automatic control. Using joystick control (from ;witches, dual three-axis controllers, or six-axis controller) 
the operator can input motion commands in parallel with inputs from sensor-based automatic controllers. 

During 1986, TRSS was enhanced to enable simultaneous control of both PUMA manipulators. Current studies 
are addressing dual-arm telerobotlc assembly tasks. In order to manually control both arms simultaneously, 
task-referenced control options were Implemented. This enables the operator to define a task reference pont 
for translational and rotational control inputs, and then the manual commands are transformed and the 
manipulators driven, with respect to the reference point. For example. If a long truss member gasped by 
both arms the task reference point could be selected between (or beyond) the two grasp points, and both arms 
would be commanded to move the truss member in a coordinated motion referenced to that point. 
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Despite the success In simultaneous control of two manipulators and In the teleoperator assembly of columns 
and nodes In 1SRL using TRSS, a great deal of work remains to reliably automate space asseafcly tasks. For 
example, small Inaccuracies In manipulator dimensions can cause disturbance forces when two arms try to move a 
column. Hybrid control methods based on force sensing have been proposed for this closed chain task, but have 
not yet been proven in TRSS. Multiple arm coordination and control Is an active research area. 


5. Virtual Architecture 


Even simple telerobotlc tasks require a large amount of software. In the traditional approach to 
telerobotlc programing, the level of detail Is minimized by encapsulating the most tedious programming Into a 
real-time executive whose only user /programmer access Is via a robot programming language. A serious nm.ta- 
tion of this approach is the lack of portability. An alternative approach Is one In which the most tedious 

ceMios 1 n t h. e L_gAl;dlJ^ pragg - STO lafl^ 

manipulating well-defined data structures via the language of the user s choice. By doing so, manipulators, 
sensors, and controls are Integrated In such a manner that conventional programming techniques (and 
tlonal programmers) can be applied to telerobot programming. The Idea Is to provide a contnon reference model, 
or virtual architecture, for TRSS and other applications In ISRL. 


A virtual architecture, designated the Teleoperator and Robotics Testbed (TART), Is implemented on the VAX 
11-750. Concurrency, mutual exclusion, and signaling mechanisms are provided by the VMS operating system. 
FORTRAN was chosen as the Implementation language because Its data structures are a subset of those present in 
most modern computer languages and It Is the language familiar to the largest number of laboratory personnel. 
It should be emphasized that a program written In any language can be used for algorithm devel OP"*"* so long is 
the TART data structures can be represented and manipulated. To date, languages used Include FORTRAN, Pascal 
and C. The Implementation suffers all the Inefficiencies of high-level languages In general (and FORTRAN In 
1 particular); however, experience has shown these limitations are moderated by the additional flexibility and 
freedom offered by a general programming language. 

TART Is a layered product In which each successive layer provides additional value to the sy* 1 **- 
Currently, five layers are Implemented: (1) user, (2) system, (3) scheduling, (4) communications, and (5) 

servo/sensor. The lowest four layers of TART are designed for minimal use of user resource quotas, but 
accomplish most of the detailed programming and error checking required by user applications. The algorithms 
used insure robust, device Independent processing and control of sensor and actuator systems, and they allow 
user-layer applications to concentrate on algorithm development. 


The lowest level of the TART system, the servo/sensor, contains processes to perform sensor Interpretation 
and actuator control functions. Generally, the algorithms for these operations execute on dedicated 
controllers. An example of a servo/sensor process Is the timer-driven routine Implementing base-referenced 
Cartesian moves for a manipulator. This process is responsible for accepting and converting setpoint and 
trajectory information from user processes to torque commands for each manipulator joint. Health and status 
information Is returned to user processes through the Intermediate layers of the TART architecture. 


To Insulate higher levels from the peculiarities exhibited by specific hardware devices, the second .ayer 
of TART, dedicated to real-time conmunl cations and data conversion, Is provided. This process examines the 
TART data structures and communications channels for state modifications. In the case where a change is 
detected In the data structures, a command is formed and transmitted to the appropriate hardware device. In 
the case where a communications channel requests service, the Information is accepted and converted to a TART 
data representation. 

The next system layer, real-time scheduling, is implemented as an asynchronous, demand-driven P^ess. 
TART differs from most robot controller executives In that scheduling of most routines Is accomplished 
implicitly through the TART data structures. Each user and system-defined instantiation of a data type 
contains a field specifying an evaluation function to be applied to the data. This field either Is null, 
contains the starting virtual address of a subroutine local to the Image, or contains the name of an entry 
olnt in an installed, sharable image [8]. The first case implies that the data Is constant and no evaluation 
is performed. In the later case, the specified image is dynamically loaded into virtual memory before applying 
tne evaluation function. The frequency of evaluation is specified in a timer field. Timer values are 
expressed in VMS format and may be null, delta, or absolute. A null entry implies (mediate, one time evalua- 
tion; delta times imply periodic application of the evaluation function; and absolute times imply one t we 
evaluation at some future time. This is similar to a mechanism for automatic update of matrices implemented by 
Hayward, but applied uniformly to all data types [9]. Scheduling is implemented via the VMS asynchronous 
system trap (AST) mechanism with the address of the data structure containing the entry address as the first 
argument of the AST argument list. 

The fourth level of the architecture, the system level, provides mechanisms for the addition, deletion, 
replacement, allocation, and deallocation of user and system defined data structures to the simulation data 
base. For all TART applications, predefined system data structures are contained in an installed conwon 
memory, or mailbox, on the VAX in order to minimize overhead in data access. This mailbox is global to all 
processes running on the VAX and users may directly access system data structures when appropriate. Typical of 
system defined data structures is the structure which defines the PUMA manipulator contained in ISRL. Once a 
data structure is added to the simulation data base, maintenance of the structures and scheduling of its 
evaluation functions is done by the real-time scheduling layer. 
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Presently* the top layer of the system Is designs tod the user level. While all TART data **7™' U J3* 
global usare are encouraged to use only the TART -defined system level aechanlsas for modification of data 
structures. Functions fo? aost of the coaaon data transformations ^ 0 r 0 ^T C L 
provided at the user level as a library of subroutines. Teaplates and definitions for TART data structures may 
be Included via language-specific constructs In any application. 

6. Future Plans 

Two future goals are to Incorporate more ROBSIM capabilities Into TRSS, and to Improve the ® f 

ISRL and TRSS with other RASA telerobotlc facilities. The TART architecture will support ^*®^ v *** 

Langley Is supporting research In the application of concurrent processing techniques to Improve the computa- 
tion speed of P the manipulator dynamics. Computing system dynamics at actuator update ^ uld 1 i n 5 r !J2l Sd 
fidelity and performance of TRSS as well as reduce Interactive delays when structural flexibility Is modelled 

In ROBSIM. 

To Increase the level of compatibility of ISRL with the~0AST Technology Demonstration program at J PL, the 
PDP-11 subsystem computers are being supplemented with OEC MIcroVAX II enters. This 

multitasking utilization of a high-speed serial communications bus as employed by JPL. The mcrovax system 
will retain* compatibility with the parallel bus structure presently used In ISRL, and enable evaluation of 
serial and parallel bus communications for telerobotlc tasks. 

7. Concluding Remarks 

The objective of automation research at Langley Research Center Is to advance tKhmlojy In the i arm of 
the mechanisms, controls, sensing, and operator Interface required by space telerobotlc tasks. Two of th« 
principal tools used to accomplish this objective are a Mgh-fldell ty simulation of robotic sy**J m * 
and a real-time man-1 n-the-loop Teleoperator and Robotics System Simulation (TRSS). TRSS is based . 

level virtual architecture Implemented in the Intelligent Systems Research Laboratory. Greater flexibility and 
reduced development time have been realized with this system architecture. 
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